Defining and debugging module completeness graphs based on interactive user interface checklist elements

ABSTRACT

Certain aspects of the present disclosure provide techniques for encoding rules defining a completeness of input, including receiving a first input comprising one or more tuples, wherein a tuple of the one or more tuples comprises one or more fields associated with an operation, one or more indicators, and one or more modifiers; receiving a second input associated with the one or more tuples; providing, to a knowledge engine, the first input and the second input; receiving, from the knowledge engine, a result based on the first input and the second input; determining, based on the result, a first symbol associated with a first tuple of the one or more tuples; and displaying the first symbol, wherein the first symbol indicates whether the first tuple is complete.

INTRODUCTION

Aspects of the present disclosure relate to defining, generating, andtesting a completeness graph of a knowledge engine through input to auser interface (UI) in order to determine if all required fields of amodule received required user input in an application.

Knowledge engines contain programs called modules that may execute oneor more operations by referencing one or more graphs of the knowledgeengine, such as calculation graphs and completeness graphs. Generally, acalculation graph of a module may be used to perform the one or moreoperations, and a completeness graph of a module may be used to ensurethat all input needed for the operations to be completed has beenreceived.

Knowledge engines allow experts to encode rules for programs in amachine-readable way. The various types of rules about a topic definewhat data is needed under certain circumstances and how to calculateresults from that data. These rules can be bundled together into a setof rules that define the module.

However, conventional methods for adding and/or modifying completenessgraphs of modules are resource-intensive (e.g., time, money, computing,personnel, etc.) and can require a deep understanding of how to writeprogramming code. For example, when defining a completeness graph byconventional methods, a user must specifically account for each andevery way to complete the application, which can implicate millions ofoptions. Consequently, it will not only take the user extreme amounts oftime to account for every way, but will in turn lead to extra processingrequirements, and thus, power requirements, as more code must beanalyzed, more input must be processed, and more processing cycles mustbe run. Where large and/or complex knowledge engines are used, thecomputational resources necessary to complete conventional methods issignificant.

Therefore, a solution is needed that can overcome the shortcomings ofthe conventional methods so as to allow for users to more easily definecompleteness graphs and reduce the processing requirements of executingthe completeness graphs.

BRIEF SUMMARY

Certain embodiments provide a method for encoding rules defining acompleteness of input. The method generally includes encoding rulesdefining a completeness of input, comprising: receiving a first inputcomprising one or more tuples, wherein a tuple of the one or more tuplescomprises one or more fields associated with an operation, one or moreindicators, and one or more modifiers; receiving a second inputassociated with the one or more tuples; providing, to a knowledgeengine, the first input and the second input; receiving, from theknowledge engine, a result based on the first input and the secondinput; determining, based on the result, a first symbol associated witha first tuple of the one or more tuples; and displaying the firstsymbol, wherein the first symbol indicates whether the first tuple iscomplete.

Other embodiments provide processing systems configured to perform theaforementioned method as well as those described here; non-transitory,computer-readable media comprising instructions that, when executed byone or more processors of a processing system, cause the processingsystem to perform the aforementioned method as well as those describedhere; a computer program product embodied on a computer-readable storagemedium comprising code for performing the aforementioned method as wellas those further described here; and a processing system comprisingmeans for performing the aforementioned method as well as those furtherdescribed herein.

The following description and the related drawings set forth in detailcertain illustrative features of one or more embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The appended figures depict certain aspects of the one or moreembodiments and are therefore not to be considered limiting of the scopeof this disclosure.

FIG. 1 depicts an example computing environment for defining acompleteness graph of a module.

FIG. 2 depicts an example user interface receiving builder inputdefining a calculation graph of a module.

FIG. 3 depicts an example user interface receiving builder inputdefining a completeness graph of a module.

FIG. 4 depicts an example user interface displaying one or more symbolsassociated with runner input, where the runner input contains therequired input to complete the module.

FIG. 5 depicts an example user interface displaying one or more symbolsassociated with runner input, where one or more lists do not need toreceive certain input in order for the module to be considered complete.

FIG. 6 depicts an example user interface displaying one or more symbolsassociated with runner input, where the runner input causes the moduleto short-circuit.

FIG. 7 depicts an example user interface displaying one or more symbolsassociated with runner input, where the runner input does not containthe required input to complete the module.

FIG. 8 depicts an example user interface displaying one or more symbolsassociated with runner input, where an “any of” indicator defines acondition where only one list out of multiple lists must be consideredcomplete for the module to be considered complete.

FIG. 9 depicts an example method of defining a completeness of a module.

FIG. 10 depicts an example method of debugging a completeness graphbased on builder input.

FIG. 11 depicts an example processing device that may be configured toperform the methods described herein.

FIG. 12 depicts an example computing device.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe drawings. It is contemplated that elements and features of oneembodiment may be beneficially incorporated in other embodiments withoutfurther recitation.

DETAILED DESCRIPTION

Aspects of the present disclosure provide apparatuses, methods,processing systems, and computer-readable mediums for defining anddebugging a completeness graph of a module by using fields within themodule and using rules and modifiers to specify which input is requiredwhen conditions are satisfied.

A user can define a module that can perform one or more operations, aswell as how those operations are completed, by coding rules about theoperations and their completeness. For example, certain applicationsallow users to define calculation graphs and completeness graphs for themodule's operations to be executed by a knowledge engine, where thecalculation graph is defined as which operations will be performed whenthe application receives certain data, and the completeness graphdefines which of that certain data that the application needs to receivein order to fully execute the operations of the module. When executingthe calculation graph and completeness graphs, the knowledge enginetakes the rules of the calculation and completeness graphs and appliesthem to a set of input data to produce a set of results.

By conventional methods, the calculation graphs and completeness graphswould be defined by writing software code, and thus, would requireextensive knowledge of coding as well as extensive amounts of time forwriting the code. Additionally, to define a completeness graph in such away would require not only the writer to account for every possibleinput combination for the operation, which results in more code to beprocessed in order to run the completeness graph, and thus makingdefining and using a completeness graph for large operations extremelytime consuming and resource intensive. However, a completeness graphthat has not been properly defined will cause the module to runimproperly, and thus, in order to have a properly working module, a userwould need to spend said time and resources. Embodiments describedherein leverage the technical capabilities of modern devices to providean improved method of defining the calculation and completeness graphsfor an operation, and therefore overcome the technical and experientiallimitations with conventional methods.

Generally, embodiments described herein may leverage various technicalcapabilities to provide an improved user interface for defining acompleteness graph through various elements. In particular, embodimentsherein provide a technical solution to the technical problem ofincreased resource requirements of coding a module, increased processingrequirements for performing the operations of a module, and increasedprocessing requirements for processing the completeness graph of themodule.

Embodiments described herein may receive, for example, builder inputreferencing one or more fields of a module and defining conditions ofwhich operational input (called “runner input”) to the fields of themodule must (or must not) satisfy in order for the module to beconsidered complete, meaning that all data required to fully process anoperation has been provided to the fields of the module. Thus, the usermay define the completeness graph with builder input referencingspecific fields of the module and choosing other builder input thatdefines conditions, and running an artifact based on those fields andconditions against the knowledge engine. Beneficially, the user canchoose specific indicators and modifiers that define the conditionswhich allows the server to generate an artifact based on the conditions,allowing the knowledge engine to check the input against the conditioninstead of checking for every possible input, thus decreasing the sizeof the artifact than what it would have been if the user writing a codethat must account for every possible input. Thus, the processingrequirements of generating and executing the artifact are significantlylower than if they were based on other codes due to decreased size, andthus increases the efficiency of the server and knowledge engine as wellas decreasing the amount of memory required to generate and execute theartifact.

In these ways and others as described herein, the embodiments hereinimprove upon conventional methods of defining and implementingcompleteness graphs by requiring significantly less space and lessprocessing for creating and processing the completeness graphs.

As described in more detail below, receiving builder input referencingfields and defining conditions of completeness not only allows for moreefficiently defining the completeness graph but also reducing theprocessing requirements and saving memory by decreasing the amount ofpossibilities that the knowledge engine must account for when executingan artifact based on the builder input. For example, the server andknowledge engine allow for a user to debug the completeness graph thathas been defined by the builder input to an associated user interface. Auser may define the completeness graph by referencing fields anddefining conditions through indicators and modifiers input to a userinterface, and then may allow the completeness graph to be tested to seeif it performs as intended. In doing so, the knowledge engine executesan artifact created by the server based on the user's definition,instead of all possible combinations of input, therefore savingprocessing power and memory space. The server can then use the knowledgeengine's execution to indicate to the user whether the completenessgraph performs as intended. Thus, the method of defining a completenessgraph described herein allows the server to reduce the processingrequirements of each test by only checking whether certain conditionsare satisfied and therefore, allowing the server and knowledge engine torun more efficiently than conventional methods allow.

Example System of Computing Devices and Server for Defining CompletenessGraphs

FIG. 1 depicts an example computing environment 100 for defining one ormore completeness graphs and calculation graphs.

As illustrated, the example computing environment 100 includes a server102 interacting with computing devices 104 and 106. In this depictedexample, the server further includes a module builder 110, a knowledgeengine 120, a client UI component 130, and application programminginterfaces (APIs) 140. In some embodiments, the knowledge engine may beimplemented outside of server 102, such as implemented on multipledevices in a distributed computing environment, or as a cloud service,or the like.

A computing device, such as computing device 104 or computing device106, can include a computer, laptop, tablet, smartphone, a smart device,a virtual machine, a container, or other types of computing device asare known.

In some cases, the computing device 104 can be associated with a user.The user can, via the computing device 104, input data to define one ormore modules (e.g., builder input as described below with respect toFIGS. 2-8 ). For example, the computing device 104 can access and/orreceive UI views from a module builder 110 in a user application and/oran editing application that may receive builder input. In some cases,the UI views may be tabular. In those cases, the UI views may consist ofmultiple rows and columns containing fields that may receive input data.In other embodiments, the UI views may not be tabular, and may bepresented through a graphical editing application. Through the UI views,the user can input the builder input that defines a completeness graphwhich the server 102 can use to generate an artifact and verify ifrunner input for performing the one or more defined operations includesthe required input necessary to execute the one or more operations. Inthis embodiment, a computing device sends one or more runner inputs tothe server 102. In some embodiments, the computing device 104 may sendthe runner input to the server 102 for debugging the completeness graph.In other embodiments, the computing device 106 may send input from aclient user as the runner input to the server 102. In yet anotherembodiment, the server 102 may use input stored on the server 102 orotherwise received at the server 102 as the one or more runner inputs.

As shown in this example, the server 102 includes module builder 110,knowledge engine 120, client UI component 130, and APIs 140.

In this depicted example, the module builder 110 includes UI component112, generating component 114, and feedback component 116.

The UI component 112 may generate UI interfaces that may be provided toone or more computing devices and may receive builder input and/orrunner input from the one or more computing devices. The builder inputprovided may be used to define at least one completeness graph for amodule, and may further define at least one calculation graph for themodule. Examples of builder input defining the completeness graph aredescribed below with respect to FIG. 3 , whereas examples of builderinput defining the calculation graph are described below with respect toFIG. 2 .

The UI component 112 may additionally contain data related to one ormore calculation graphs and/or completeness graphs that can be used todefine certain aspects of input to the UI interfaces. For example, theUI component 112 may contain data listing all fields of the module, allindicators, and all modifiers that may be used as input to a UIinterface, such as user interface 300 of FIG. 3 . In some embodiments,such data may be stored in a network or on a separate server that may beaccessed by module builder 110.

The generating component 114 can generate a set of artifact filescorresponding to the module (e.g., a “module code”) defining thecalculation graphs, completeness graphs, and UI views for the modulebased on the builder input to the module interface. In some embodiments,each of the calculation graph, completeness graph, and client UI viewfor the module correspond to a respective artifact file. Further, eachartifact may be an XML, files, JSON files, or other structured datafiles. The generating component 114 can further include an algorithmthat reviews the builder input to the module interface to create amodule code based on the builder input. Each module code or parts of themodule code may be stored in the knowledge engine at calculation graphs122, completeness graphs, 124, and/or artifacts 126. In otherembodiments, the artifacts may be stored in a network or another serverthat may be accessed by server 102 to retrieve artifacts that will beexecuted by the knowledge engine.

As illustrated, knowledge engine 120 includes calculation graphs 122,completeness graphs 124, and artifacts 126. As previously noted, theknowledge engine 120 may also receive one or more artifact filesdefining one or more calculation graphs, completeness graphs, and UIviews that it may store in artifacts 126. With the artifact files, theknowledge engine 120 may generate and/or update the calculation graphsand completeness graphs which may be stored in calculation graphs 122and completeness graphs 124, respectively. In some embodiments, theknowledge engine 120 does not update the calculation graphs orcompleteness graphs, and instead, the module builder 110 updates theartifacts defining the calculation graphs and completeness graphs, whichthe knowledge engine 120 uses to generate new calculation graphs andcompleteness graphs. The module builder 110 may have access to theknowledge engine 120 and may retrieve calculation graphs andcompleteness graphs if needed to define the one or more UI viewsprovided to a computing device, such as computing device 104. In thisdepicted example, the knowledge engine is part of the server 102. Inother embodiments, the knowledge engine may be implemented elsewhere.

As shown in this depicted example, the server 102 further includesclient UI component 130. The client UI component 130 may generate one ormore client views to provide a computing device, such as computingdevice 104 or computing device 106. The client UI component 130 may alsoreceive runner input from the computing devices 104. The client UIcomponent 130, after receiving runner input, may provide the runnerinput to the server 102 so that the server 102 can generate an artifactbased, at least in part, the runner input for the knowledge engine 120to execute against the appropriate calculation graph and completenessgraph. The knowledge engine 120 may provide results of executing theartifact against the calculation graph and completeness graph to one ormore of the computing devices. The results of executing the artifactagainst the calculation graph would indicate specific calculations madebased on the received runner input and operations associated with thecalculation graph. The results of executing the runner input against thecompleteness graph would indicate if the received runner input containedall input required in order to complete the operation of the module.

Server 102 further includes APIs 140, which may be used by the server102 in order to determine if the module has been properly defined. Insome embodiments, a user can call one or more APIs of APIs 140corresponding to a particular artifact file of the module code in orderto cause knowledge engine 120 to execute the artifact file (e.g.,calculation graph, completeness graphs, and/or a client UI view) basedon received runner input. In some embodiments, an API can cause theknowledge engine 120 to run the calculations of the calculation graphbased on runner input, cause the knowledge engine 108 to run thecompleteness graph to make sure all required runner input was input tothe collection interface, and/or cause the knowledge engine 108 todisplay an associated client UI view to the user. In another embodiment,an API can cause the module builder 110 to generate client UI viewsafter the executing certain artifact files. The server may receive aresult based on that execution and display one or more symbols based onthat result, so that the user can debug a completeness graph. In someembodiments, the explanation may be further defined by input received atthe server 102. By running the APIs, the user can make sure that alldesired rules and regulations of the module collection have beenimplemented and all necessary data has been input.

Example User Interface for Defining a Calculation Graph of a Module

FIG. 2 depicts an example user interface 200, which is one of the UIviews that may receive builder input for defining a calculation graph ofa module or runner input to test that calculation graph. In someembodiments, the user interface 200 is provided by a server, such asserver 102 of FIG. 1 , to a computing device, such as computing device104 of FIG. 1 .

As illustrated, this particular user interface 200 includes a tabularinterface for a user to enter builder input regarding a calculationgraph (e.g., how to perform a new and/or modified operation) of themodule or the runner input to test the module. User interface 200includes one or more tuples, such as a particular set of user interfaceelements (e.g., a set of cells), that may receive builder input. Inputto each tuple may define a node of the knowledge graph and/or anoperation associated with the node. In this depicted example, the tuplesmay be columns of cells and rows of cells, however, columns of cells androws of cells are exemplary tuples, and other tuples may be used inother embodiments. For example, each column of the user interface mayhave a textual description of the data contained in the column (e.g.,“role” and “type”). Each row can receive builder input representing adescription of a respective node in the calculation graph or anoperation to be performed by the node. Together, the rows representmultiple nodes of the calculation graph that will perform one or moreoperations to generate a result.

For example, a user can input builder data (e.g., “builder input”) to afirst cell (e.g., cell 214) in a first row to define a first node with anode description of “User Name” (e.g., as input by the user to cell218), and builder input to a second cell (e.g., cell 216) in the firstrow to include in the description the type of data (e.g., “string”,“decimal”, “checkbox”, or “date”), to be entered in a UI view. The usermay additionally input a field name (e.g., “Name”) for the first node incell 220, which may further be used when inputting information definingthe completeness graph, as described with respect to FIG. 3 . The fieldname may refer to a field of the module that may receive test data input(e.g., “runner input”) in order to perform an operation of the module.Lastly, the user interface 200 may receive runner input to be runagainst the calculation graph, such as data related to the node (e.g.,“John Smith” in cell 222).

In some embodiments, the builder input may be chosen from a dropdownmenu or may be typed in by the user. In other embodiments, certain cellsmay be automatically filled in by the server and cannot receive userinput. In some embodiments, the runner input may be typed into anappropriate cell of the user interface 200.

The user may further input data to the remaining fields in the userinterface (e.g., the remaining fields in set of cells 204, set of cells206, set of cells 208, set of cells 210, and set of cells 212), whereeach row of the remaining cells corresponds to a node in the calculationgraph. The server may then generate an artifact file defining thecalculation graph, which may be used to execute the builder input andrunner input to user interface 200 against a knowledge engine, such asknowledge engine 120 of FIG. 1 .

In some embodiments, the builder input to user interface 200 may bestored by the server, and later accessed by the server (e.g., by UIcomponent 112), in order to define other user interfaces of the module,such as user interface 300 of FIG. 3 .

Example User Interface for Defining a Completeness Graph of a Module

FIG. 3 depicts an example user interface 300, which is one of the UIviews that may receive builder input for defining a completeness graphof a module. In some embodiments, the user interface 300 is provided bya server, such as server 102 of FIG. 1 , to a computing device such ascomputing device 104 of FIG. 1 .

As shown, the user interface 300 is tabular and has one or more tuplesof cells that may receive builder input defining the completeness graphof the module. Similarly to user interface 200, the tuples in thisdepicted example are rows and columns, however, rows and columns areexemplary, and other types of tuples may be used. In other embodiments,the user interface may be a graphical user interface or a user interfacethat receives plain text using a domain specific language. Afterreceiving builder input to at least one cell, the server may create anartifact based on the builder input that may be executed by theknowledge engine in order to determine the completeness of input to themodule. For example, in some cases, a user may define the completenessgraph by using builder input related to the input to another userinterface view, such as the user interface 200 of FIG. 2 , and theserver may use the builder input to user interface 300 to create anartifact that may be run against the knowledge engine. In someembodiments, the server may create and run the artifact against theknowledge engine through an API (e.g., an API of APIs 140), which insome cases may be done in response to a user selection. Additionally,the server may also receive builder input for debugging the module orclient input to the fields of the module in order to determine thecompleteness of input, which an artifact may further be based on.

Each column may be associated with a type of data of the builder input.For example, a set of cells may cover input required for the entiremodule to be complete or for a subset of the module, called a “list” offields, to be complete (e.g., cells in column A having data referencinga field of the module, a specific set of fields referenced by a list, orthe entire module). As another example, a set of cells may containreferences specific fields or specific lists (e.g., cells in column Breceiving builder input to each cell of set of cells 308, set of cells314, and set of cells 320 referencing specific fields of the module, aswell as set of cells 304 and set of cells 306 defining lists). Further,a set of cells may include one or more indicators denoting whether thereis a condition associated with a field in the same row and when thatcondition needs to be satisfied (e.g., cells of column C having dataindicating whether or not a field must always receive input in set ofcells 310, set of cells 316, and set of cells 322). Also, certain cellsmay also receive a modifier defining the condition for the field in thesame row (e.g., cells in columns D-G defining conditions of when a fieldmust receive input, such as set of cells 312, set of cells 318, and setof cells 324). Thus, together, a combination of one or more indicatorsand modifiers may define a condition and when that condition applies.

In FIG. 3 , the cells of row 3 receive builder input to each of cells3B-3F defining when a field (e.g., cell 3B defining the “Email” field ofthe module “Legal Age”) requires input for the module to be consideredcomplete. More specifically, cell 3B of row 3 denotes the field of themodule (e.g., “Email”, corresponding to a field in set of cells 220 asdescribed with respect to FIG. 2 ), cell 3C of row 3 denotes that the“Email” field “MUST BE ASKED WHEN” a condition is satisfied, where thecondition is defined by cells 3D-3G of row 3 (e.g., when the “Name”field in set of cells 220 did not receive corresponding input, andtherefore is “UNKNOWN”).

As noted above, rows of user interface 300 may also receive builderinput indicating that the entire module or a certain list of the moduleis required to receive runner input in order for the module to becompleted. For example, set of cells 302 corresponding to row 1 (alsoreferred to as “module row 302”) defines that the entire module named“Legal Age” is complete when all of the fields defined by set of cells308 receive input, based on whether conditions defined by indicators inset of cells 310 and modifiers in set of cells 312 are satisfied.

Similarly, builder input to the rows may define that one or more fieldsreferenced by one or more lists are required to receive runner input inorder for the list to be completed based on whether certain precedingconditions are satisfied. For example, set of cells 304 corresponding torow 10 (also referred to as “list row 304”) may receive builder inputthat defines that the “Parent” list is required to receive runner inputto each referenced field in the list, as defined by set of cells 314, inorder for the list to be completed, subject to conditions defined by anindicator of cell 10C and modifiers of cells 10D-10G. Additionally, therows corresponding to the list may have their own cells of fields,indicators, and modifiers that define when the fields of the rows mustreceive runner input. For example, row 13 may receive input definingthat the “Parent” list is complete when the field referenced by cell 13B(“Psign”) receives input, subject to certain conditions defined by theindicators in set of cells 316 and the modifiers in set of cells 318. Incertain embodiments, when defining a list, no indicators or modifiersare necessary to define a condition for the list, and instead, may bedefined in the cell referencing the list. For example, in thoseembodiments, cell 10A may receive builder input defining that all of thereferenced fields of the list must receive runner input (e.g., cell 10Awould receive builder input showing “List—All of”) or alternatively,that only one of the referenced fields of the list must receive runnerinput (e.g., cell 10A would receive would receive builder input showing“List—Any of”). After defining the fields of a list, builder inputreferencing the list may be input to certain cells (e.g., cells 6B and7B). Thus, lists may be used as builder input to define when other listsor the module may be complete, which allows the module or lists to bebroken up into logical chunks.

As an additional example, the user interface 300 may receive builderinput that defines that the “Guardian” list, as denoted by input to cell17B, requires runner input to each referenced field of the list definedby set of cells 320, in order for the list to be completed, where theinput must be received subject to certain conditions as defined by theindicator of cell 17C and modifiers of cells 17D-17G.

Thus, the builder input to the cells of user interface 300 defines whencertain fields of the module as defined in user interface 200 of FIG. 2must receive runner input, and may define that certain fields mustreceive input subject to not only the conditions defined in the rowreferencing the field, but other conditions as well. In doing so, thebuilder input may link certain fields or lists together, making certainfields only require runner input when multiple conditions are satisfied.

For example, in order for row 21 to be considered complete, the fieldreferenced by input to cell 21B (e.g., “Gday”) of row 21 must receiverunner input based on the condition defined by the indicator of cell 21Cand modifiers of cells 21D-21G. However, row 21 is within the “Guardian”list as defined in row 17, and thus, “Gday” must also receive runnerinput for the condition defined by the indicator of cell 17C andmodifiers of cells 17D-17G to be satisfied. Additionally, the “Guardian”list is also input to set of cells 308, and thus, for the “Legal Age”module to be considered complete, the fields defined by the “Guardian”list (e.g., fields input to set of cells 320) must only receive input ifthe condition defined by the indicators of cells 7C and 8C and modifiersof cells 7D-7G and 8D-8G is satisfied.

In this way, a user may use builder input defining when a module hasreceived the necessary input subject to certain indicators andidentifiers, which may include defining when one or more lists or fieldshave received the necessary input as well.

Referencing Fields within the Module

The builder input to each cell in set of cells 308, set of cells 314,and set of cells 320 may reference one or more fields (as defined inuser interface 200 of FIG. 2 ) of the module that require runner input,subject to certain conditions associated with the row that referencesthe field, as described below. The input to a cell may reference a fieldof the module (e.g., “Name”, “Email”, “Bday”, etc., as defined by set ofcells 220 of FIG. 2 ). In this depicted example, a field of the modulethat requires input may be defined by a user in a separate userinterface view, as described with respect to user interface 200 of FIG.2 .

The builder input to one or more cells may further indicate what fieldsof the module must receive runner input in order for the module itselfto be considered complete. Like the lists, the module itself may beindicated in its own row of cells (a “module row”), for example, modulerow 302 for the “Legal Age” module. In some embodiments, the module orlist names may be defined by text input of the user. In some embodiment,like the other rows, module row 302 may receive builder input definingits own condition through indicators and modifiers (e.g., the indicatorof cell 1C and the modifiers of cells 1D-1G), meaning that the modulemay only be considered complete when the condition is satisfied. Inother embodiments, the module row 302 may not receive any builder inputdefining its indicators and modifiers, and may instead be complete whenall referenced fields have received input if the associated conditionsare satisfied. In this depicted example, the fields that must receiveinput for the module to be complete are shown as the cells directlybeneath the row defining the module (e.g., the “Legal Age” moduleindicated by module row 302 receives input that certain fields of themodule defined by set of cells 308 must receive runner input for the“Legal Age” module to be complete, subject to conditions defined byassociated indicators and modifiers).

The builder input to one or more cells may further define a list offields of the module (e.g., “List-Parent” and “List-Guardian”). Thecells defining a list of fields of the module may receive builder inputreferencing one or more fields of the module that must receive runnerinput in order for the list to be complete, based on whethercondition(s) of the row referencing each field are satisfied.Additionally, the list itself may be indicated in its own row of cells(a “list row”), for example, list row 304 for the “Parent” list or listrow 306 for the “Guardian” list. Like other rows, the cells of the rowthat indicates the list may receive builder input defining indicatorsand modifiers of the row. In this depicted example, the fields of thelist are defined by the cells directly beneath the list row (e.g., the“Parent” list indicated by row 304 includes all fields defined bybuilder input to the set of cells 314). In other embodiments, the fieldsof the list may be defined in a separate user interface view or may bein a separate location within the user interface 300.

After a list is defined within user interface 300, for example, by listrow 304 and set of cells 314 along with the corresponding indicators andmodifiers, the list may be referenced like a field of the module bybuilder input to other cells of user interface 300. For example, the setof cells 308 includes cell 6B, which corresponds to the “Parent” list.Thus, in order for the “Legal Age” module to be considered complete, the“Parent” list must also be considered complete, based on the conditionsassociated with row 1 (defining when the entire “Legal Age” module iscomplete), row 6 (defining when the “Parent” list specifically must becompleted for the “Legal Age” to be complete), row 10 (defining when theentire “Parent” list is complete), and the rows 11-15 (defining wheneach field defined by the set of cells 318 must receive for the “Parent”list to be complete).

In this depicted example, the fields that must receive runner input aredefined input to a drop down menu. For instance, in response to a userselection, a cell in the set of cells 308, set of cells 314, or set ofcells 320 may display a drop down menu of all available fields and liststo receive runner input (e.g., all fields defined by the set of cells220 as described with respect to FIG. 2 as well as lists defined in userinterface 300). In other embodiments, text input to cells correspondingto one or more of the available fields may be used to indicate whichfields must receive runner input.

Further, in this depicted example, there is no limit to the amount oftimes a certain field chosen from the available fields may be used indefining the completeness of the module or lists of the module. Forexample, the field “Psign” as indicated by builder input to cell 13Cmust receive runner input for the “Parent” list to be completed,however, the user could also define other lists that required Psign toreceive runner input in order for the list to be completed.

Indicators Labelling Fields in the User Interface

In FIG. 3 , rows reference fields of the module through input to cellsin the set of cells 308, set of cells 314, and set of cells 320 may beconsidered complete subject to conditions defined in part by indicatorsinput to cells of the set of cells 310, set of cells 316, and set ofcells 322, respectively. An indicator defines when a field referenced ina cell of the same row or a corresponding row must receive runner inputbased on whether a condition defined by the corresponding modifiers ofthe same row is satisfied. For example, the field “Email” as defined bycell 3B must receive runner input only in certain circumstances asdefined by the indicator of cell 3C for the module to be consideredcomplete.

In this depicted example, each row contains an indicator (e.g., input toa corresponding cell in column C) corresponding to the field referencedby the builder input. Further, in this depicted example, the indicatorof each cell in set of cells 310, set of cells 316, and set of cells322, is defined whether or not the cell receives input. For example, ifa cell receives builder input, the indicator is defined by that input,but if the cell does not receive input, the indicator becomes a defaultindicator (a “blank indicator”) in this example. In other embodiments,rows may not require input to a cell defining an indicator correspondingto the referenced field. In another embodiment, the cells must receiveinput in order to define the indicator of the cell.

In some embodiments, the indicator may be chosen from a set ofindicators. For example, an indicator may be “WHEN” as shown byindicator of cell 6C, or may be “MUST BE ASKED WHEN” as shown byindicator of cell 3C. Additional examples of indicators in the set ofindicators include “MUST BE ASKED”, “is complete when”, and a blankindicator.

The “WHEN” indicator indicates that, for a module or list referencing acorresponding field to be considered complete, the referenced fieldrequires runner input only “WHEN” the condition defined by correspondingmodifiers is satisfied. Thus, in this depicted example, the “Legal Age”module will only be considered complete “when” the “Parent” list asdefined by cell 6B receives required runner input, as long as thecondition defined by the modifiers of cells 6D-6G is satisfied.

The “MUST BE ASKED WHEN” indicator indicates that, for a module or listreferencing a corresponding field to be complete, the referenced fieldmust be presented to a user when the condition defined by thecorresponding modifiers is satisfied. Thus, in the depicted example, the“Legal Age” module will only be considered complete when the “Email”field, as referenced by cell 3B, is displayed to a client user when thecondition defined by the modifiers of cells 3D-3G is satisfied. In someembodiments, the corresponding field may be complete even when thereferenced field does not receive any runner input from the user.

The “MUST BE ASKED (BLANK IS ALLOWED)” indicator indicates that areferenced field must be displayed to a client user in all cases. Thefield corresponding to the indicator will be considered complete as longas the referenced field is displayed, regardless of whether thereferenced field receives runner input.

The “is complete when” indicator indicates that a module or listassociated with the indicator is complete only when referenced field orfields receive input, as long as the conditions defined by the modifiersassociated with those certain fields are satisfied. In some embodiments,a blank indicator may also indicate that a module or list associatedwith the indicator is complete only when referenced field or fieldsreceive input, as long as the conditions defined by the modifiersassociated with those certain fields are satisfied.

In this depicted example, the blank indicator indicates that either nobuilder input was received at the cell or the builder input to the cellindicated a blank indicator. The blank indicator further indicates thatthe referenced field always requires input for the field, list, ormodule, regardless of any conditions defined by the modifiers associatedwith the certain field. In this depicted example, the “Name” fielddefined by cell 2B is associated with a blank indicator input to cell2C, indicating that the “Name” field must always receive input for the“Legal Age” module to be complete, regardless of if any conditions aredefined by the modifiers of cells 2D-2G. In some embodiments, if themodifiers of cells 2D-2G are not blank modifiers but the indicator ofcell 2C is a blank indicator, the server may display an error messageindicating that the user should adjust the current builder input.

A field associated with a blank indicator may not require builder inputfor the entire module to be complete if the list referencing the fieldis not required to be complete. For instance, in this depicted example,the “Parent” list only requires certain input “when” the conditiondefined by modifiers input to cells 6D-6G is satisfied. Further, fieldsreferenced by the “Parent” list are also associated with a blankindicator (e.g., in cell 11C). Thus, if the “Parent” list does not needto be considered complete for the module to be considered complete, thefields referenced by the list that are associated with the blankindicator may not require input for the “Legal Age” module to becomplete.

In some embodiments, a referenced field may be associated with two ormore conditions. In those embodiments, the field may require runnerinput only if one condition of the two or more conditions is satisfiedor all conditions of the two or more conditions are satisfied, asdescribed below.

A user may input an indicator to a cell through a selection to drop downmenu. For instance, in response to a user selection, an indicator inputto the set of cells 310, set of cells 316, or set of cells 322 maydisplay a drop drown menu of all available indicators to receive builderinput. In other embodiments, text input to the cells may be used todefine which indicators are input to the cells.

Further, in this depicted example, there is no limit to the amount oftimes a certain indicator may be used when defining the completeness ofa module or lists of the module.

While certain indicators may be shown and defined in this depictedexample, those indicators are exemplary, and other indicators may beused when defining a completeness graph.

Modifiers Defining Conditions for when Referenced Fields Must ReceiveInput

In FIG. 3 , the builder input to cells in the set of cells 308, set ofcells 314, and set of cells 320, referenced fields that must be completewhen conditions as defined by builder input to cells in the set of cells312, set of cells 318, and set of cells 324 are satisfied. One or moremodifiers input to cells define a condition of a row referencing acorresponding field. For example, the field “Email” as defined by cell3B must be presented when the “Name” field equals “UNKNOWN” (as shown bymodifiers in cells 3D-3F) in order for the module to be complete.

In this depicted example, certain cells of each row receive certainmodifiers as builder input corresponding to the field and the indicatorof the row. Further, in this depicted example, the modifiers input tothe cells of a row define a condition that, in conjunction with theindicator of the row, indicates when the field associated with the rowmust receive runner input. For example, the fields associated with the“Parent” list, as referenced by input to cell 6B, must only receiverunner input “when”, as defined by the indicator of cell 6C, thecondition of defined by modifiers of cells 6D-6G. However, when thecondition defined by the modifiers is not satisfied, the indicator doesnot apply, and the referenced field or fields do not require runnerinput for the module to be considered complete. Further, in thisdepicted example, the condition defined by the input to cells 6D-6Gshows that the indicator is only applicable when the value associatedwith the “Age” field of the module “is less than” “18”. Thus, “when” thevalue associated with the “Age” field is “less than” “18”, as defined bythe modifiers of cells 6D-6G and the indicator of cell 6C, the field(e.g., the “Parent” list) of the module referenced by cell 6B willrequire input. Alternatively, “when” the value associated with the “Age”field is not “less than” “18”, the “Parent” list does not need to becomplete for the module to be considered complete. In other embodiments,the values of specific modifiers may be defined in another interface andreferenced in user interface 300. For example, instead of the modifier“18” in cell 6F, the modifier “AdultAge” could be used in cell 6F, where“AdultAge” may be defined as a constant in user interface 200.

A modifier may be chosen from a set of modifiers. Further, there may bemultiple types of modifiers.

A first type of modifiers may correspond to fields of the module, suchas fields referenced in set of cells 210 as described with respect toFIG. 2 (e.g., the “Age” field as input to cell 6D). A second type ofmodifiers may correspond to operators, such as operators determining ifone value “IS LESS THAN”, “IS GREATER THAN”, “EQUALS”, “DOES NOT EQUAL”,“IS LESS THAN OR EQUAL TO”, or “IS GREATER THAN OR EQUAL TO” anothervalue. While the text describing the operators have been describedabove, the equivalent mathematical symbols may also be input as amodifier.

A second type of modifiers may further include operators determining ifone value is “ONE OF” an array of values, is “NOT ONE OF” an array ofvalues, or is a blank or unknown value.

A third type of modifier may define a value to which another value needsto be compared. In some embodiments, the value of the field may bedefined by runner input to a referenced field. The value may be anumerical value, a string value, a Boolean value, or another type ofvalue.

A fourth type of modifier indicates whether certain conditionsassociated with the referenced field must be satisfied. For example, thefourth type of modifier may be “and”, as shown by modifier of cell 7G inthis depicted example, indicating that both a first condition defined atleast in part by builder input to cells 7D-7F “and” a second conditiondefined by the builder input to cells 8D-8F must be satisfied in orderfor the corresponding field to require input. In another example, thefourth type of modifier may be “or”, indicating that either a firstcondition “or” a second condition must be satisfied in order for thereferenced field to require runner input in order to be consideredcomplete. In this depicted embodiment, only two conditions are linked bythe “and” modifier input to cell 7G. In other embodiments, any number ofconditions may be related by the “and” or “or” modifiers. For example, aset of conditions within parentheses may be linked by “and” modifiers,and those conditions within the parenthesis may be separated fromanother condition or a set of conditions by “or” modifiers, so that onlyone set of conditions must be satisfied in order for the referencedfield to be considered complete.

Certain types of modifiers may further correspond to conditions definingwhen a referenced module or a list of fields is complete.

For example, a fifth type of modifiers may include the “all of”modifier, which indicates that a module or list is complete when “allof” the listed fields corresponding to the module or list receive runnerinput, as long as the corresponding conditions associated with thefields are satisfied. In this depicted example, modifiers of cells 1D,10D, and 17D show that all fields listed below require input subject toindicators and modifiers for the “Legal Age” module, the “Parent” list,and the “Guardian” list, respectively. Alternatively, the fifth type ofmodifiers may also include the “any of” modifier, which indicates that amodule or list is complete when only one of the referenced fieldscorresponding to the module or list receives runner input, as long asthe corresponding conditions are satisfied (as shown by cell 11F of FIG.8 ).

Additionally, a sixth type of modifier may also correspond to conditionsdefining when a referenced module or a list of fields is complete. Forexample, the sixth type of modifier may include the “these are complete”modifier, as shown in modifiers of cells 1E, 10E, and 17E, whichindicates that a module or list is complete when all of the fieldslisted for the module or list receive input, as long as thecorresponding conditions are satisfied. In some embodiments, a blankmodifier in column E may also indicate that a module or list is completewhen all of the fields listed for the module or list receive input, aslong as the corresponding conditions are satisfied.

While certain modifiers may be shown and defined in this depictedexample, those modifiers are exemplary, and other modifiers may be usedwhen defining a completeness graph.

Thus, the combination of the indicators and the modifiers for each rowdefines when corresponding fields must receive input in order forcertain modules or lists to be considered complete.

Short-Circuit Cells

The builder input defining the completeness graph of a module may causethe module or list to “short-circuit”, meaning that the knowledge engineexecuting the completeness graph does not need to evaluate the rest ofthe runner input to fields of the module or list.

In FIG. 3 , cell 5B displays “THIS LIST WILL SHORT CIRCUIT” subject towhether an associated condition is satisfied. Therefore, in the depictedexample, the list will short-circuit if the condition defined by themodifiers is satisfied, and thus, if the knowledge engine evaluates row5 and determines that the condition defined by modifiers 5D-5G issatisfied, the knowledge engine will short-circuit the module andconsider it complete regardless of the required runner input of to thefields referenced by rows that have not yet been evaluated. Thus, if theknowledge engine short-circuits evaluating the module at row 5, theknowledge engine will not need to further evaluate the runner input tofields corresponding to fields referenced by cells 6B and 7B. In someembodiments, the knowledge engine may continue to evaluate the input torows or lists that have not yet been evaluated, and provide resultsindicating if the row, list, or module would have been complete if theknowledge engine had not short-circuited. In other embodiments, theknowledge engine may stop evaluating the module or list and will notprovide results based on the unevaluated lists, thus saving time andprocessing power because it has not been required to evaluate the extrainput. For instance, in those embodiments, if the module itselfshort-circuits before a list is evaluated, the entire list may not beevaluated at all by the knowledge engine.

In some embodiments, the rows with conditions associated withshort-circuiting are evaluated before the other rows within acorresponding list or module. In those embodiments, if the list ormodule short-circuits because the condition is satisfied (or notsatisfied), the knowledge engine does not evaluate any other rows of thelist or module, and thus, saves times and processing power by evaluatingfewer rows than normally required.

Additionally, a seventh type of modifier may correspond to a conditiondefining that a referenced field must receive required runner input whena module or list short-circuits or does not short-circuit. For example,the condition defined by the modifier may only require that a first listreceives input to corresponding fields if a second list hasshort-circuited, or alternatively, not short-circuited, for the moduleto be considered complete. Thus, the builder input may define conditionsthat make a first list or the module short-circuit based on if a secondlist has short-circuited.

Thus, when a referenced field receives or does not receive certainrunner input, the knowledge engine may short-circuit evaluating themodule or list, thereby saving time and processing power by onlyevaluating a portion of the input when determining completeness of theinput.

Dead-Ends Occurring when a Field does not Receive Required Input

In some embodiments, when debugging the module or evaluating runnerinput, the knowledge engine may “dead-end” evaluating the module inresponse to the runner input. “Dead-ending” occurs when the knowledgeengine has partially evaluated the runner input and determined that amodule or list cannot be completed due to runner input to one field or avalue associated with that field, regardless of runner input to otherfields. For example, a dead-end may occur if input to a cell in a rowreferences “THIS LIST WILL ONLY BE COMPLETE” when a correspondingcondition of the row is satisfied, but the runner input does not satisfythe condition. Thus, no matter what other input may be required for thelist or module to be complete, the list or module cannot be completed,and the list or module will dead-end.

Therefore, in this depicted example, dead-ending would occur if runnerinput to the fields of the module caused the value associated with the“Page” field to be less than “18”. Since the condition of row 15, whichis associated with the “Page” value, would not be satisfied, the“Parent” list could never be complete, regardless of runner input to theother referenced fields. Thus, the “Parent” list would dead-end. In someembodiments, when the knowledge engine determines that a list or moduledead-ends, it does not evaluate the other referenced fields of the listor module, and returns a result indicating that it did not evaluatethose other referenced fields.

Thus, by inputting certain fields, indicators, and modifiers to thecells of the user interface 300, a user may define the required runnerinput for a module to be completed. The combination of the indicatorsand the modifiers may define conditions that determine exactly whenfields require runner input or do not require runner input, when a listor module will short-circuit, and when a list or module may dead-end.Further, lists of fields may be defined by the user, which may haveassociated conditions of their own defining when the lists need to bereceive required runner input. After receiving all of the builder inputdefining the completeness graph, the server may create an artifact basedon the builder input and run the artifact against the knowledge enginewhen determining if the runner input completes the module.

Debugging a Module Based on the Fields, Indicators, and Modifiers andDebugging Input

With the user interfaces associated with the module, a user may input(1) builder input defining the calculation graph, such as the input touser interface 200 of FIG. 2 , (2) builder input defining thecompleteness graph, such as the input to user interface 300 of FIGS. 3 ,and (3) runner input used to test the completeness of the module basedon the runner input and the builder input defining the completenessgraph, such as described below with respect to FIGS. 4-8 . The runnerinput used to test the completeness graph may be input in a userinterface for defining the calculation graph, such as user interface200, a user interface for defining the completeness graph, such as userinterface 300, or a separate user interface. Runner input may beconsidered any input to a field of a module that is used or referencedwhen executing one or more operations of the module. In someembodiments, the runner input may be input to a user interfaceassociated with the module (e.g., user interface 200 or user interface300) for debugging the module. In other embodiments, the runner inputmay be input to a user interface displayed to a user on a client device.

Based on the builder input defining the completeness graph and therunner input to the fields of the module, a server, such as server 102of FIG. 1 , may create an artifact to execute against a knowledgeengine, such as knowledge engine 120 of FIG. 1 . The artifact may thenbe executed against the knowledge engine, which may return a resultbased on the completeness of the module with respect to the builderinput defining the completeness graph and runner input, as describedwith respect to FIG. 1 .

Based on the result of the completeness of the builder input, the servermay display one or more symbols associated with the completeness of oneor more the rows in the user interface by determining if a referencedfield, list, or module is complete based on if runner input satisfied anassociated condition. Each symbol may indicate one or more aspectsassociated with the completeness of a tuple of input.

In order to communicate the aspects associated with the completeness ofthe module, the server may display the symbols (e.g., such as symbols446 a, 446 c-e, 446 a, and 450 and sets of symbols 446 b and 448 b ofFIG. 4 ) in the user interface that received builder input with extrarows and columns for displaying the symbols and runner input. Forexample, user interface 300 of FIG. 3 , which received the builderinput, may display extra rows and columns with the symbols and runnerinput in order to show the one or more aspects that were determined.Thus, the symbols and runner input may be displayed in the same userinterface that received the builder input (e.g., as described withrespect to user interface 400 of FIG. 4 , user interface 500 of FIG. 5 ,user interface 600 of FIG. 6 , user interface 700 of FIG. 7 , and userinterface 800 of FIG. 8 ). The rows may further be highlighted with acolor of a plurality of colors, where the highlighted color of the rowis associated with the symbol displayed for the row.

In some cases, the extra rows and columns with the symbols and runnerinput may be displayed in response to a user request received by theserver. Further, in those cases, the extra rows and columns may behidden in response to another user request received at the server. Inother embodiments, the symbols and runner input may be displayed in anentirely separate user interface or user interface view.

As noted above, the user interface may display one or more symbolsassociated with the completeness of the one or more tuples, whichindicate the completeness of an associated field, list, or module. Inthe embodiments described below, the tuples are rows of cells of a userinterface. In other embodiments, the tuples may be a different type ofcontainer, such as a column of cells, a box of cells, or anotherarrangement of cells. In another embodiment, the tuples may not includecells and may comprise certain a graphical arrangement. In otherembodiments, the tuples may be defined in a text interface.

The symbols may further indicate one or more aspects of correspondingfields, lists, or modules based on the builder input and the inputdefining the completeness graph, and may be displayed next to a rowassociated with that aspect and corresponding field, list, or module.For example, the symbols may indicate that the runner input to the fieldreferenced by the row is complete because the runner input satisfies acondition of the row, as described with respect to FIGS. 4-8 , or mayindicate that a row is not complete because the runner input to a fieldreference in the row does not satisfy the condition of the row, asdescribed below with respect to FIGS. 5-8 .

As another example, the symbols may indicate whether a module or listreferenced by the row has short-circuited based on whether the runnerinput associated with the module or list satisfies the condition of therow, as described below with respect to FIGS. 4-8 , and may alsoindicate if a first row is considered complete due to a short-circuitregardless of runner input to a referenced field, as described furtherbelow with respect to FIG. 6 . The symbols may also indicate whether arow is complete based on whether a field, list, or module was notevaluated due to a dead-end, as described with respect to FIG. 7 , ormay indicate whether a first row is considered complete because the rowdid not need to be evaluated based on a satisfied condition of a secondrow, as described further below with respect to FIGS. 5 and 8 .

The server may analyze the result returned by the knowledge engine anddetermine symbols to display based on the completeness of the runnerinput and the one or more aspects in order to help the user determinewhether to adjust the builder input, the runner input, or both. In somecases, after displaying the symbols indicating the completeness of themodule, the server may receive new builder input or new runner input. Inresponse to a user request received through an interface element, theserver may generate an artifact based on the new input and receive a newresult from the knowledge engine, and may update what symbols aredisplayed in the interface. It should further be noted that the aspectsdetermined and symbols as described with respect to FIGS. 4-8 areexemplary, and other aspects and symbols may be determined and shown.

Even further, in order to determine if the runner input completes thefields, lists, or module, the server may use an API (e.g., an API ofAPIs 140 of FIG. 1 ) in order to provide the artifact generated based onthe builder input and runner input to the knowledge engine. An API mayfurther return the result indicating the completeness and one or moreaspects from the knowledge engine to the server.

Example Debugging a Module that has Received the Required Input

FIG. 4 depicts an example user interface 400 displaying one or moresymbols based on the results of executing an artifact based on thebuilder input defining a completeness graph, such as the builder inputto user interface 300 of FIG. 3 , and runner input to the fields of themodule, that may be input to a user interface such as user interface 200of FIG. 2 . For example, the runner input may be based on input data tothe set of cells 212 of FIG. 2 .

In this depicted example, based on the result of the completeness of thebuilder input returned by the knowledge engine, the user interface 400displays one or more symbols (e.g., such as symbols 446 a, 446 c-e, 446a, and 450 and sets of symbols 446 b and 448 b) associated with thecompleteness of one or more rows 1-25. The rows of user interface 400may correspond to the rows of other user interfaces, such as userinterface 200 and/or user interface 300. In this depicted example, therow 5 references data from row 2 of user interface 200, where the fieldof the module (e.g., the “Name” field) is defined, and row 2 of userinterface 300 where the completeness requirements for the field aredefined.

Additionally, some rows of user interface 400 refer to specific lists ofthe module that are defined in another user interface, such as userinterface 300. In this depicted example, row 4 refers to the “Legal Age”module defined by row 1 of user interface 300 (e.g., module row 302 ofFIG. 3 ), and further includes the corresponding indicators andmodifiers. Additionally, row 13 refers to the “Parent” list defined byrow 10 of user interface 300 (e.g., list row 304 of FIG. 3 ), andfurther includes the corresponding indicators and modifiers. Evenfurther, row 20 refers to the “Guardian” list defined by row 17 of userinterface 300 (e.g., list row 306 of FIG. 3 ), and further includes thecorresponding indicators and modifiers.

The symbols 446 a, 446 c-e, 446 a, and 450 and sets of symbols 446 b and448 b are determined by the server based on the result of executing anartifact against a knowledge engine, such as knowledge engine 120 ofFIG. 1 , where the artifact is based on the completeness of runner input(e.g., the input data to set of cells 220 of FIG. 2 ) as defined by thebuilder input defining the completeness graph (e.g., the input data touser interface 300). For example, the server may determine the symbolsbased on a table mapping portions of a result to certain symbols. Thus,the knowledge engine checks to see if the one more fields, lists, andmodules referenced by the rows are complete by executing the artifact.In this depicted example, the knowledge engine would start by checkingto see if the “Legal Age” module of as referenced by row 4 is completeby checking if the runner input satisfies the conditions of the rows5-11. In doing so, the knowledge engine may check if the referencedfields and lists associated with the module, such as the fields andlists referenced by the builder input to set of cells 408, are complete.In some cases, this requires the knowledge engine to evaluate a listwith respect to more than one condition associated with the list. Thus,in this depicted example, the “Parent” list of row 9 only needs to beevaluated when the runner input causes the “Age” value to be “less than”“18”; however, if the runner input causes the “Age” value to be “lessthan” “18”, the knowledge engine must evaluate the completeness of the“Parent” list. In those cases, the “Parent” list “is complete when” “allof” its referenced fields (e.g., as defined by set of cells 418) receiverunner input, as long as the conditions defined by the indicators (e.g.,as defined by set of cells 420) and modifiers (e.g., as defined by setof cells 422) that are associated with the rows are satisfied.

In this depicted example, the symbol 446 a indicates that the runnerinput to the fields associated with the rows 5-11 contains the requiredinput because the runner input satisfies certain condition of the rows5-11, and therefore, each row is considered complete. Thus, the serverdetermined that, based on the result of the knowledge engine, the “LegalAge” module and the “Parent” list were complete because all associatedfields received required input based on the associated conditions.

For example, based on the result, the server determined the “Legal Age”module received all required input at least in part because the “Name”,“Email”, and “Bday” fields of the module received the required inputfrom the builder data that is indicated by set of symbols 446 b. Thebuilder input data to the “Name” (e.g., “John Smith”), “Email” (e.g.,“jsmith@intuit.com”), and “Bday” (e.g., “4/20/2003”) fields is furtherdisplayed in set of cells 416 in order to show the user what input wasused to determine the completeness of those fields. In particular, the“Name” field and the “Bday” field are associated with a checkmark symbolbecause the runner input received satisfied the condition associatedwith the row (e.g., the conditions of rows 5 and 7 are associated withblank indicators and modifiers, meaning that the “Name” and “Bday”fields must receive runner input under all circumstances). The “Email”field is instead associated with the triangular symbol because thecondition associated with the row not satisfied, and thus, the “Email”field did not need to receive runner input, and even did not need to bepresented to the user. In other embodiments, the “Email” may also beassociated with the checkmark symbol, as it still received input and isconsidered complete, even though the associated condition was notsatisfied. Thus, each of rows 5-7 are considered complete, and theserver displays set of symbols 446 b indicating the completeness ofthose rows. If a row was not considered complete, the server woulddisplay an “X” symbol for that row indicating that the row was notcomplete.

After determining the “Name”, “Email”, and “Bday” fields are consideredcomplete, the server determines that the “Legal Age” module does notshort-circuit based on the result because the “Age” value associatedwith the runner input (e.g., “17.97”) is not “less than” “13”. Thus, theserver displays symbol 446 c based on that determination, which is asingle rectangle representing that the module did not short-circuitbased on the condition of row 8.

Next, the server determines whether the “Parent” list, as shown in cell9D, needs to receive input based on the result. Thus, since the “Age”value associated with the runner input (e.g., 17.97 as shown in cell 9J)is “less than” “18” (as defined by the modifiers of cells 9F-9I), the“Parent” list requires certain input. Therefore, based on the result,the server determines one or more aspects about the “Parent” list inorder to determine whether it has received the required input.

For example, a determined aspect may be that the “Parent” list mustreceive input for the “Legal Age” module to be complete, as shown by row9. The required input for the “Parent” list is defined by rows 13-18;thus, in order for the “Parent” list to be considered complete, “all of”(as shown by the modifier input to cell 13F) the fields referenced inset of cells 418 must receive input based on whether the condition ofthe corresponding row is satisfied, and additionally, the conditionassociated with row 13 must also be satisfied. Based on the result, theserver can determine that the “Pname”, “Pemail”, “Psign”, and “Pday”fields all received runner input (e.g., as shown in cells 426). Further,the server may also determine that the conditions for each row of rows13-17 have been satisfied because the rows only contained blankindicators and modifiers, meaning that any input to the field wouldsatisfy the condition. Lastly, the server may further determine that thelist may still be complete because the “Page” value is not less than 18,as indicated by the condition of row 18. Thus, the server displays setof symbols 448 b to indicate that the rows 14-18 have received therequired input. Additionally, based on the result, the server determinesthat row 13 is complete based on the runner input satisfying thecondition associated with row 13.

After determining that the “Parent” list has received the requiredinput, the server determines whether the “Guardian” list needs toreceive input based on the result and displays an associated symbol.Since the “Age” value associated with the runner input (e.g., “17.97” asshown by cell 10K) is “less than” “18”, but the “Pname” field receivedrunner input and therefore is not “UNKNOWN”, the server determines thatthe “Guardian” list does not need to receive input because at least oneof the conditions associated with row 10 was not satisfied. Thus, theserver displays symbol 446 e (e.g., a triangular symbol) to denote anaspect indicating that the “Guardian” list was complete, regardless ofany input to the fields of the “Guardian” list.

Further, while the user interface 400 contains the list defining whatfields must receive input for the “Guardian” list to be complete, theserver determines an aspect based on the result that the “Guardian” listdoes not need to be analyzed because one of the conditions associatedwith row 10 was not satisfied. Thus, in this depicted example, theserver displays symbol 450 (e.g., a triangular symbol) corresponding torow 20 in order to denote that the “Guardian” list is consideredcomplete because at least one of the conditions associated with the“Guardian” list was not satisfied (e.g., and thus, the referenced fieldsof the “Guardian” list did not need to receive input in order to beconsidered complete). In other embodiments, the server may furtherdisplay symbols, such as an “?”, for each row of rows 21-25 denotingthat each specific field within the “Guardian” list was not evaluated,and thus, the server did not determine whether the fields of the rowrequired runner input based on their associated conditions.

Thus, based on the result returned by the knowledge engine and theaspects determined by the server, the server displays symbol 446 aindicating the completeness of the module, including the completeness ofthe lists and fields within the module.

In this depicted embodiment, the knowledge engine evaluates thecompleteness of the rows in the order that the rows are shown (e.g.,from top to bottom). In other embodiments, the knowledge engineevaluates and rows, in a specific order defined in another userinterface, such as user interface 300. For example, in that embodiment,the knowledge engine may wait to evaluate a specific field or list untila separate list has been evaluated (e.g., the knowledge engine mayevaluate row 9 on whether the “Parent” list should be evaluated, andsubsequently, may evaluate rows 13-18 to determine if the “Parent” listis complete, before evaluating row 10 on whether the “Guardian” listshould be evaluated). Thus, the result for each of those embodiments maybe different, and may cause the server to display one or more extrasymbols than are shown in the depicted embodiment. In yet anotherembodiment, the knowledge engine evaluates the completeness of rows inan arbitrary order.

Further, in this depicted embodiment, certain conditions may define aconstant that the value of a field may be compared to, such as “UNKNOWN”of cell 6H or 11H, “13” of cell 8H, and “18” of cell 9H or 10H. In otherembodiments, instead of constants, the input to those cells mayreference fields. In those embodiments, the knowledge engine will usethe value of the referenced field when evaluating the condition of thatrow, and will indicate that in the result.

FIG. 5 depicts an example user interface 500 displaying one or moresymbols based on the results of executing an artifact based on thebuilder input defining a completeness graph of a module and runner inputto the fields of the module. For example, input to user interface 300 ofFIG. 3 may define the completeness graph, while builder input may beinput to user interface 200 of FIG. 2 . In this depicted example, userinterface 500 is based on the builder input to user interface 300 aswell as builder input defining an extra “Address” list defined by fieldsof the module, as shown by row 12 and rows 28-31. As similarly discussedwith respect to FIG. 4 , the server generates an artifact based on thebuilder input and the runner input, which the knowledge engine mayexecute to return a result to the server indicating one or more aspectsof the rows defined by the builder input.

As shown in this depicted example, in order for the module to beconsidered complete, (1) the “Name”, “Email”, and “Bday” fields must beconsidered complete based on the conditions of rows 5-7, and (2) the“Parent” list, the “Guardian” list, and the “Address” list must receiverequired input in order to be considered complete if the module does notshort-circuit.

As shown by the builder input data in set of cells 518, the “Name”,“Email”, and “Bday” fields received runner input, and thus the serverdetermines that the fields are complete based on the result returned bythe knowledge engine. In this depicted example, the server displays acheckmark symbol for both the “Name” and the “Bday” field because theassociated conditions are satisfied, while the server displays thetriangular symbol because the associated condition is not satisfied andthus the “Email” field did not need to receive input. Since the fieldsare complete, the server determines the set of symbols 550 bcorresponding to rows 5-7, indicating the fields of the rows receivedall required input, and thus, the rows are considered complete.

Next, the server determines a symbol for the row 8 based on if themodule short-circuited or not. Based on the result, the serverdetermines that the module did not short-circuit because the “Age” valueassociated with the builder input (e.g., 18) is not “less than” “13”,and therefore, does not satisfy the condition defined by the indicatorsand modifiers of row 8. Thus, the server determines the single rectangleas the symbol to display for row 8, and displays symbol 550 c toindicate the module did not short-circuit based on the evaluation ofthat row.

The server further evaluates whether or not the “Parent” and “Guardian”lists are considered complete based on the result. Based on the result,the server determines that the “Parent” list does not need to receiveinput in order to be considered complete because the “Age” value isassociated with the runner input is not “less than” “18”, and therefore,the condition of row 9 is not satisfied. Thus, since the row referencesa list that is complete regardless of whether it is evaluated, theserver displays the triangular symbol 550 d to denote that the “Parent”list is complete regardless of whether it received input. Further, theserver also displays question mark symbol 552 a corresponding to row 14,which defines that the “Parent” list is complete subject to the “Pname”,“Pemail”, “Psign”, and “Pday” fields receiving input, in order tofurther show that the “Parent” list does not need to receive input inorder to be considered complete because the condition associated withrow 14 was not satisfied. Thus, even though the builder input did notreceive runner input to the “Pname”, “Pemail”, “Psign”, and “Pday”fields (as indicated by set of cells 528, where the runner input foreach field is “UNKNOWN”) and the server displays set of symbols 552 b todenote that those fields did not receive runner input, the “Parent” listis still considered complete.

Additionally, based on the result, the server determines that the“Guardian” list does not need to receive input in order to be consideredcomplete because the “Age” value associated with the runner input is not“less than” “18”, and therefore, does not satisfy the conditions of rows10 and 11. Thus, since the row references a list that is completeregardless of whether it is evaluated, the server displays thetriangular symbol 550 e to denote that the “Guardian” list is completeregardless of whether it received input. Further, the server displaystriangular symbol 554 a to indicate the condition of row 11 was alreadydetermined to not be satisfied, and the “Guardian” list only needed toreceive input to be considered complete if both conditions weresatisfied. The server also displays questions mark symbol 554 acorresponding to row 21, where row 21 defines that the “Guardian” listis complete subject to the “Gname”, “Gemail”, “Gsign”, and “Gday” fieldsreceiving input, in order to further show that the “Guardian” list doesnot need to be evaluated because it does not need to receive input inorder to considered complete since the condition associated with row 11was not satisfied. Thus, even though the runner input did not containinput to the “Gname”, “Gemail”, “Gsign”, and “Gday” fields (as indicatedby set of cells 538, where the runner input for each field is “UNKNOWN”)and the server displays set of symbols 554 b to denote that those fieldsdid not receive input, the “Guardian” list is still considered complete.

Since the “Parent” list and “Guardian” list were determined to becomplete regardless of receiving input, and thus do not affect thecompleteness of the module, the module only further needs to receiveinput to the “Address” list as indicated by row 12, based on thecondition of row 12. Since the indicators and modifiers of row 12 areblank indicators, the “Address” list must always be evaluated unless themodule either short-circuits or dead-ends before evaluating the addresslist in order for the condition to be satisfied.

As the server determines, based on the result, that the module neithershort-circuited nor dead-ended (e.g., because the condition associatedwith row 8 was not satisfied), the server also evaluates whether the“Address” list is considered complete. Based on the result, the serverdetermines that certain fields referenced by the “Address” list requireinput in order to be considered complete because the indicators andmodifiers of row 12 are blank, indicating that there is no condition forwhich the “Address” list does not require runner input. Thus, the serverfurther evaluates, based on the result, whether the fields referenced inrows 29-31 received the required runner input and displays symbols setof symbols 556 b indicating that the rows are complete since they havereceived the required input based on associated conditions. In addition,since all required input for the “Address” list was received, the serverfurther displays symbol 550 g corresponding to row 12 and symbol 556 acorresponding to list row 508 to indicate that the “Address” list iscomplete.

Thus, as indicated by symbol 550 a, the “Legal Age” module is consideredcomplete because it has received all required runner input. Morespecifically, the server determined, based on the result, that the“Name”, “Email”, and “Bday” fields all received the required input, asindicated by set of symbols 550 b. Further, the server determined, basedon the result, that the module did not short-circuit based on thecondition associated with row 8, as indicated by symbol 550 c. Theserver also determined, based on the result, that the module did notneed to evaluate either the “Parent” list or the “Guardian” list becausethey would be complete regardless of whether they received input, anddisplayed symbols 550 d and 550 e. Lastly, the server determined, basedon the result, that the “Address” list received the required input, asindicated by symbol 550 g.

Example of Debugging a Module where the Runner Input Caused aShort-Circuit

FIG. 6 depicts an example user interface 600 displaying one or moresymbols 608 a, 608 c-e, 610 a, and 612 a, and sets of symbols 608 b, 610b, and 612 b based on the results of executing an artifact based on thebuilder input defining a completeness graph, such as the input to userinterface 300 of FIG. 3 , and runner input to the fields of the module,that may be input to a user interface such as user interface 200 of FIG.2 .

In this depicted example, as similarly described with respect to FIG. 5, the “Name”, “Email”, and “Bday” fields of rows 5-7 received therequired input as shown by the runner input in set of cells 614. Thus,based on the result returned by the knowledge engine, the serverdetermines the fields referenced by rows 5-7 are complete and displaysset of symbols 608 b.

Next, as similarly described with respect to FIG. 5 , the serverdetermines a symbol for the row 8 based on if the module short-circuitedor not. Based on the result, the server determines that the module didshort-circuit because the “Age” value associated with the builder input(e.g., “10” as shown in cell 8J) is “less than” “13”, and therefore,satisfies the condition of row 8. Thus, the server determines anddisplays phi CO symbols 608 a and 608 c to indicate the moduleshort-circuited.

As described above with respect to FIG. 3 , when a moduleshort-circuits, the remaining rows of the module that have not beenevaluated do not need to receive input in order for the module to beconsidered complete. Thus, in the depicted example, the serverdetermines the “Parent” list and the “Guardian” list do not need toreceive input in order for the module to be considered complete anddisplays the double rectangular symbols 608 d and 608 e to indicate thatthe lists did not need to be evaluated. Further, in this depictedexample, the server displays symbol 610 a and symbol 612 a correspondingto rows 13 and row 20, respectively, to indicate that the listsassociated rows did not have to be evaluated.

In this depicted embodiment, the set of rows 14-18 and set of rows 21-25associated with the “Parent” list and “Guardian” list, respectively,were evaluated even though they were not required to be evaluated. Inother embodiments, the server may not evaluate set of rows 14-18 and setof rows 21-25, which may reduce processing requirements and save time,and thus, power, used by the server.

Example of Debugging a Module where the Runner Input Caused a Dead-End

FIG. 7 depicts an example user interface 700 displaying one or moresymbols 712 a, 712 c, 714, and 716, and sets of symbols 712 b and 712 dbased on the results of executing an artifact based on the builder inputdefining a completeness graph of a module and runner input to the fieldsof the module. For example, input to user interface 300 of FIG. 3 maydefine the completeness graph, while runner input may be input to userinterface 200 of FIG. 2 .

As shown in this depicted example and as similarly described withrespect to FIGS. 4-5 , in order for the module to be consideredcomplete, (1) the “Name” and “Email” fields must receive required inputsubject to the indicators and modifiers of rows 5-6 if the module doesnot short-circuit, (2) the “Age” value must not equal “UNKNOWN” asdefined by the condition of row 7 if the module does not short-circuit,and (3) the “Parent” list and the “Guardian” list must be consideredcomplete if the module does not short-circuit.

In this depicted example, as similarly described with respect to FIGS.5-6 , the “Name” and “Email” fields referenced by rows 5-6 received therequired input as shown by the builder input in set of cells 710. Thus,the server determines and displays set of symbols 712 b indicating thatthe “Name” and “Email” fields are complete.

However, unlike as described with respect to FIGS. 5-6 , the “Bday”field did not receive input, and thus, the “Age” value associated withthe runner input cannot be calculated, making it “UNKNOWN”. Thus, theknowledge engine generates a result indicating that the “Age” value isunknown, and based on the result, the server displays the “UNKNOWN”value of cell 7B in set of cells 710. Since the row 7 does not referencea field in cell 7D, and instead indicates “THIS LIST WILL BE COMPLETEONLY”, the module dead-ends if the associated condition of the row isnot satisfied. Since the “Age” value associated with the runner inputequals “UNKNOWN”, the condition is not satisfied. Thus, the serverdetermines that the module dead-ended and displays symbol 712 c (e.g.,the “X” symbol) to indicate that the module dead-ended because thecondition of row 7 was not satisfied.

In addition, since the module dead-ended and cannot be completed basedon the received runner input, the server also determines that the “LegalAge” module is incomplete, and displays symbol skull-and-crossbonessymbol 712 a in order to indicate that the module is incomplete due to adead end.

In this depicted example, after determining that the module isincomplete, the server does not evaluate the remaining rows 8-10 anddisplays set of symbols 712 d as well as symbols 714 and 716 (e.g., “?”symbols) in order to indicate that those rows were not evaluated due tothe dead-end caused by row 7. In other embodiments, the server may stillevaluate the rows that did not need to be evaluated due to the dead-end,and may still display symbols indicating whether the fields or listswould have been complete if the module had not dead-ended.

Thus, in this depicted example, as indicated by symbol 712 a, the “LegalAge” module has not received all required input and is consideredincomplete. More specifically, since the “Age” value associated with therunner input did not satisfy the condition of row 7 and the moduledead-ended, as indicated by symbol 712 c, the “Legal Age” module couldnot be completed.

Example of Debugging a Module where the Builder Input Includes an “anyof” Modifier

FIG. 8 depicts an example user interface 800 displaying one or moresymbols 850 a, 852 a, 854 a, and 856 a, and sets of symbols 850 b, 852b, 854 b, and 856 b based on the results of executing an artifact basedon the builder input defining a completeness graph, such as the input touser interface 300 of FIG. 3 , and runner input to the fields of themodule, that may be input to a user interface such as user interface 200of FIG. 2 . In this depicted example, user interface 800 is based on theinput to user interface 300 where the “Legal Age” module requires the“Adult” list to be completed instead of both the “Parent” list and the“Guardian” list to be completed, based on whether associated conditionsare satisfied.

As shown in this depicted example, in order for the module to beconsidered complete, (1) the “Name”, “Email”, and “Bday” fields mustreceive required input subject to the indicators and modifiers of rows5-7, and (2) the “Adult” list must be considered complete if the moduledoes not short-circuit. In order for the “Adult” list to be consideredcomplete at least one of the “Parent” list or the “Guardian” needs to beconsidered complete, as shown by the indicator of cell 11E and modifierof cell 11F (e.g., “any of”).

As shown by the builder input to cells 818, the “Name”, “Email”, and“Bday” fields received builder input, and thus the server determinesthat the fields are complete and displays the appropriate symbols.Further, based on the result, the server displays the appropriate symbolafter determining that the module did not short-circuit because the“Age” value associated with the builder input (e.g., 18) is not “lessthan” “13”, and therefore, does not satisfy the condition associatedwith the row.

The server then evaluates the “Adult” list based on the result returnedby the knowledge engine. Thus, according to the condition of row 11,only one of the “Parent” list and the “Guardian” list needs to becompleted in order for the “Adult” list to be considered complete. Inthis depicted example, based on the result, the server determines thatthe “Guardian” list is complete based on the builder input indicated inset of cells, and displays the appropriate symbols. Thus, since at leastone of the “Parent” list and the “Guardian” list is satisfied, theserver determines that the “Adult” list is also satisfied, and thusdisplays symbol 852 a indicating that the “Adult” list is complete. Inparticular, the server further displays set of symbols 852 b in order toindicate that the “Parent” list was incomplete and the “Guardian” listwas complete, showing that the “Parent” list may be incomplete while the“Adult” list is still complete.

In this depicted example, the server evaluates the “Parent” list eventhough only the “Guardian” list was already completed, and displays theappropriate symbols. In other examples, if the server already determinedthat the “Adult” list was completed, the server may not evaluate theother requirements of the “Adult” list.

Thus, as indicated by symbol 850 a, the “Legal Age” module has receivedall required input and is considered complete, which is based at leastin part on the “Adult” list receiving the required input.

Example Method of Defining a Completeness of a Module

FIG. 9 depicts an example method 900 of determining whether a module iscomplete based on builder input defining the completeness graph (e.g.,input to user interface 300 of FIG. 3 ) and runner input associated withthe fields of the module (e.g., that may be input to user interface 200of FIG. 2 ).

At 902, a server (e.g., server 102 of FIG. 1 ) receives a first input ata user interface, such as user interface 300. The first input (e.g.,builder input as described with respect to FIGS. 2-8 ) may define thecompleteness graph of a module associated with performing an operation,where the completeness graph indicates what data needs to be received inorder for the module to be considered complete. The first input mayinclude data referencing a set of fields of the module, as well as a setof indicators and set of modifiers defining when the fields of themodule are required to receive input when performing the operation.Thus, a combination of the set of indicators and set of modifiers defineconditions for when certain fields of the set of fields require input.

In some embodiments, each field may correspond to at least one indicatorand at least one modifier. A field may further correspond to multipleconditions defined by certain indicators and modifiers, where thecondition indicates when the field must receive input. In someembodiments, the fields, indicators, and modifiers may be chosen from adrop down menu displayed in response to a user selection, while in otherembodiments, the fields, indicators, and modifiers may be typed by theuser.

The first input may further include one or more lists of fields of themodule. For example, a list of fields may include data referencing oneor more fields of the module that must receive input for the list to becomplete, subject to the associated conditions of both the list and thefields within the list. In turn, certain lists may need to be consideredcomplete in order for the entire module to be considered complete.

In some embodiments, the first input may be based in part on inputdefining a calculation graph of the module, such as input to userinterface 200. The fields of the module that define the calculationgraph of the module may be a part of the first input defining whatfields require input in order for the module to be completed.

Certain referenced fields, indicators, and modifiers of the first inputmay indicate when the fields of the module require input, as describedabove with respect to FIG. 3 . For example, certain indicators mayindicate that a certain field, list, or module must receive inputsubject to a condition defined by associated modifiers (e.g., the “when”indicator as described with respect to FIG. 3 ″. Other indicators mayindicate that a field of the module must be presented for a user, butdoes not require input (e.g., the “must be asked” indicator as describedwith respect to FIG. 3 ). Further, certain modifiers may define acondition that each field in a set of fields must receive input for amodule or list to be complete (e.g., the “All of” modifier as describedwith respect to FIG. 3 ). Certain other modifiers may define a conditionthat only one field in a set of fields must receive input for a moduleor list to be complete (e.g., the “Any of” modifier as described withrespect to FIGS. 3 and 8 ). Some modifiers may be combined in order tocompare values associated with certain fields to other values defined bythe first input (e.g., the “IS LESS THAN”, “IS GREATER THAN”, “EQUALS”,“DOES NOT EQUAL”, “IS LESS THAN OR EQUAL TO”, or “IS GREATER THAN OREQUAL TO” modifiers as described with respect to FIG. 3 ). Certainmodifiers may connect conditions defined by other modifiers in theconjunctive or disjunctive so that a field must receive input when (1)both conditions are satisfied, or (2) either condition is satisfied,respectively (e.g., the “and” and “or” modifiers as described withrespect to FIG. 3 ).

Even further, the first input may include information indicating that amodule or list will short-circuit if an associated condition issatisfied (e.g., the “THIS LIST WILL SHORT CIRCUIT” field as describedwith respect to FIG. 3 .

Thus, the first input received at the server defines which fields of themodule require input under certain conditions, where there may bemultiple fields, and multiple conditions defined by indicators andmodifiers that indicate when those fields require input.

At 904, the server receives a second input (e.g., runner input asdescribed with respect to FIGS. 2-8 ) at a user interface. In someembodiments, the second input may be received at a different userinterface or a different user interface view, such as user interface200, than the first input. In other embodiments, the second input may bereceived at the same user interface as the first input. The second inputincludes data input to the fields of the module. In some embodiments,the data may be runner input to a user interface, such as input to userinterface 200. In other embodiments, the data may be client inputreceived from a computing device (such as computing device 106 of FIG. 1). The second input may be analyzed in order to determine if the secondinput to the module contains all required input to the fields of themodule that require input.

In some embodiments, the server may generate an artifact based on thefirst input and the second input that may be executed by a knowledgeengine. In some embodiments, the server may generate the artifact basedon just the first input. The knowledge engine may execute the artifactand return a result based on the artifact. In some embodiments, theknowledge engine may determine if the module is complete based on thesecond input and include that determination in the result. The servermay use an application programming interface (API) in order to providethe artifact to the knowledge engine.

At 906, the server receives the result from the knowledge engine. Insome cases, the server may receive the result through an API.

At 908, the server determines whether the second input contains requiredinput to the fields that require input based on the result received fromthe knowledge engine. The server analyzes the result to determine one ormore aspects associated with any combination of the module, each list ofthe module, and/or each field of the module. For example, the server maydetermine that each field of the module and each list of the module hasreceived the required input through the second input, subject to theconditions defined by the combination of indicators and modifiers of thefirst input. The server may also determine, based on the result, thatthe module did not receive the required input, and therefore is notcomplete, because the second input did not contain the required input asdefined by the combination of indicators and modifiers of the firstinput.

The server may further determine, based on the result, that the moduleor certain lists of the module short-circuited because one or moreconditions were or were not satisfied, and therefore, certain otherlists or fields did not need to be evaluated. The server may furtherdetermine, based on the result, that certain fields or lists of themodule were not evaluated because the evaluation of the module or a listdead-ended based on the conditions of the first input.

Thus, with the referenced fields and the conditions defined by theindicators and modifiers, a user may more quickly define a completenessgraph using fewer resources than conventional methods. As a result, thegenerated artifact based on the builder input and runner input requiresfewer resources to generate and less processing requirements to execute,thus increasing the efficiency and decreasing the amount of resourcesneeded to generate and test the completeness graph.

Additionally, analyzing the result, the server may display one or moresymbols indicating aspects of the result, such as if certain referencedfields received the required input based on the associated conditions.For example, the server may display symbols indicating that certainfields, lists, or modules are complete, not complete, or were notrequired to be complete, based on associated conditions. The server mayalso display symbols indicating that certain fields or lists were notevaluated because the module or a list dead-ended. Additionally, theserver may display symbols indicating if certain lists or fields werenot required to be complete based on associated conditions. Thus, afterreceiving and analyzing the result, the server may display symbols inorder to show if the completeness graph is working properly, and helprefine the definition of the completeness graph if it is properlydefined.

Note that FIG. 9 is just one example of a method, and other methodsincluding fewer, additional, or alternative steps are possibleconsistent with this disclosure.

Example Method of Debugging a Completeness Graph of a Module

FIG. 10 depicts an example method 1000 of debugging a module based onbuilder input defining the completeness graph (e.g., input to userinterface 300 of FIG. 3 ) and runner input to the fields of the module(e.g., input to set of cells 220 of FIG. 2 ). The builder input and therunner input may be input to one or more user interfaces, such as userinterface 300, of one or more computing devices (e.g., computing device104 and computing device 106 of FIG. 1 ) associated with a server (e.g.,server 102 of FIG. 1 ).

At 1002, the server receives a first input (e.g., builder input asdescribed with respect to FIGS. 2-8 ) comprising one or more fields of amodule and associated indicators and modifiers defining conditionsassociated with the one or more fields, where each field andcorresponding indicators and modifiers create an input tuple, asdescribed with respect to FIGS. 3-8 . In some embodiments, a singlefield may correspond to multiple indicators and multiple modifiers. Insome embodiments, a tuple may be a row of cells. In other embodiments, atuple may be another combination of cells or an arrangement that doesnot include cells. Together, the tuples of fields, indicators, andmodifiers define the completeness of the module, such as when certainfields require input based on the conditions defined by the indicatorsand modifiers.

Each indicator may be from a plurality of indicators and each modifiermay be from a plurality of modifiers, as described above with respect toFIG. 3 .

At 1004, the server receives a second input (e.g., runner input asdescribed with respect to FIGS. 2-8 ) associated with the one or moretuples, such as test input to one or more fields for the module. In someembodiments, the second input may be input to a separate user interface(such as user interface 200 of FIG. 2 ) than the user interfacereceiving the first input. In other embodiments, the second input may beinput to the same user interface as the first input.

At 1006, the server may provide the first input defining thecompleteness and the second input testing the completeness to aknowledge engine. The server may provide the inputs to the knowledgeengine by way of an artifact created based on both inputs, as describedwith respect to FIGS. 1-3 . The knowledge engine may receive and executethe artifact in order to determine if certain fields received input andif the input was required based on the conditions defined by the firstinput. Based on the determination, the knowledge engine may generate andreturn a result to the server indicating one or more aspects of eachtuple. For example, the result may indicate that a tuple is complete,incomplete, or another aspect of the tuple because one or more fields,one or more lists, and/or the entire module is received (or did notreceive) certain input.

At 1008, the server may receive the result based on the first input andthe second input from the knowledge engine, as described with respect toFIGS. 1-3 . The result may indicate one or more aspects of each tuple ofthe first input. In some embodiments, the result may indicate one ormore aspects of only one or more certain subsets of tuples of the firstinput. The one or more aspects for a tuple may include an indicationthat tuple received required input and therefore is considered complete,did not receive the required input and therefore is not consideredcomplete, that the input to the tuple caused the module or a list toshort-circuit, that the input to the tuple caused the module or a listto dead-end, that a tuple or a set of tuples was not evaluated, that atuple or a set of tuple did not need to be evaluated for the module tobe considered complete, or another aspect. In addition, the result mayindicate one or more aspects for a list tuple of the module.

At 1010, the server determines, based on the result, a first symbolassociated with a first tuple of the first input based on an aspect ofthe first tuple. In some embodiments, the first symbol may indicate ifthe first tuple is considered complete (e.g., because it received therequired input). Further, the server may determine symbols associatedwith other tuples of the first input based on aspects associated withthose tuples. Additionally, the server may determine more than onesymbol for a tuple if the result indicates more than one aspect for thetuple.

At 1012, the server displays the first symbol indicating an aspect ofthe first tuple, such as whether the first tuple is considered complete.In some embodiments, the server may display more than one symbol for atuple based on if the server determined more than one aspect associatedwith the tuple.

In some embodiments, after displaying the one or more symbols associatedwith the tuples based on input associated with the one or more fields,lists, and/or modules, the server may receive new builder input and/ornew test input to a user interface, such as user interface 200 or userinterface 300, and generate a new artifact based on the one or more newinputs. The server may provide the new artifact to the knowledge engine,which may return a new result indicating that the tuples and referencedfields, lists, and/or module are considered complete based on the newinputs. The server may then receive the new result, determine that thetuples and referenced fields, lists, and/or module are consideredcomplete, and display one or more new symbols indicating that thefields, lists, and/or module are considered complete.

For example, the server may display one or more symbols indicating thatcertain tuples are complete, as described with respect to FIGS. 4-8 , orthat tuples are incomplete, as described with respect to FIGS. 7-8 ,because certain input was not received. Additionally, the server maydisplay one or more symbols indicating that certain tuples caused themodule or a list of the module to short-circuit, while displaying othersymbols indicating that certain tuples did not need to be evaluatedbased on that short-circuiting, as described with respect to FIG. 6 .Even further, the server may display one or more symbols indicating thatcertain tuples dead-ended, and may also display other symbols indicatingthat other tuples were not evaluated due to that dead-end, as describedwith respect to FIGS. 7-8 . As another example, the server may furtherdisplay one or more symbols indicating that a certain tuple is completebased on an associated condition even though an associated field did notreceive input.

Thus, by using the fields and conditions defined by the second input,the server requires less time and processing power to run thecompleteness graph against the knowledge engine because it must onlycheck if certain conditions are satisfied or not satisfied rather thanhaving to check if every possible combination of input has been entered.Additionally, by displaying the symbols, the server further creates amore efficient way for a user to determine if the completeness graph hasbeen defined as desired, and as a result, can continue to test thecompleteness graph by running it against the knowledge engine multipletimes, which, as described above, requires less time and processingpower than conventional methods.

Note that FIG. 10 is just one example of a method, and other methodsincluding fewer, additional, or alternative steps are possibleconsistent with this disclosure.

Example Processing Device for Defining and Debugging a CompletenessGraph

FIG. 11 depicts an example processing device 1100 that may perform themethods described herein with respect to FIGS. 9-10 . For example, theprocessing device 1100 can be a physical processing device or virtualserver and is not limited to a single processing device that performsthe methods described above.

Processing device 1100 includes a central processing unit (CPU) 1102connected to a data bus 1112. CPU 1102 is configured to processcomputer-executable instructions, e.g., stored in memory 1114, and tocause the processing device 1100 to perform methods described above. CPU1102 is included to be representative of a single CPU, multiple CPUs, asingle CPU having multiple processing cores, and other forms ofprocessing architecture capable of executing computer-executableinstructions.

Processing device 1100 further includes input/output (I/O) device(s)1108 and I/O device interfaces 1104, which allows processing device 1100to interface with input/output devices 1108, such as, for example,keyboards, displays, mouse devices, pen input, and other devices thatallow for interaction with processing device 1100. Note that processingdevice 1100 may connect with external I/O devices through physical andwireless connections (e.g., an external display device).

Processing device 1100 further includes a network interface 1106, whichprovides processing device 1100 with access to external network 1110 andthereby external personal devices.

Processing device 1100 further includes memory 1114, which in thisexample includes a module builder 1120, a knowledge engine 1130, aclient UI view component 1140, and APIs 1150.

In this example, module builder 1120 may further include UI component1122, generating component 1124, and feedback component 1126. UIcomponent 1122 may display one or more user interfaces that may receivebuilder input defining a calculation graph, a completeness graph, or auser interface view. UI component 1122 may further receive runner inputfor testing a completeness graph. Generating component 1124 may use theuser input received at UI component 1122 to generate one or moreartifacts based on the builder input. The one or more artifacts mayfurther be based on the runner input. The feedback component 1126 mayreceive one or more results from the knowledge engine indicating one ormore aspects associated with the builder input and the runner input.Further, the feedback component 1126 may determine one or more symbolsto display based on the one or more aspects associated with the inputand may provide the one or more symbols to the UI component 1122 fordisplay on a user interface.

Further, in this example, the knowledge engine 1130 may further includecalculation graphs 1132, completeness graphs 1134, and artifacts 1136.Calculation graphs 1132 may include calculation graphs associated withmodules of the knowledge engine, and may further create or updatecalculation graphs for modules based on the artifacts generated bygenerating component 1124. Completeness graphs 1134 may includecompleteness graphs associated with modules of the knowledge engine, andmay further create or update completeness graphs for modules based onthe artifacts generated by generating component 1124. Artifacts 1136 mayreceive and store artifacts generated by generating component 1124 foruse by the knowledge engine 1130. For example, the knowledge engine mayexecute an artifact generated by generating component 1124 or stored inartifacts 1136 against a calculation graph of calculation graphs 1132and/or a completeness graph of completeness graphs 1134 in order togenerate a result.

Additionally, in this example, client UI view component 1140 maygenerate a user interface view to send to a computing device.Additionally, client UI view component 1140 may receive input from thecomputing device for use by the module builder 1120 and/or knowledgeengine 1130.

In this example, APIs 1150 may provide the artifacts generated bygenerating component 1124 to the knowledge engine based on inputreceived at the UI component 1122. Further APIs 1150 may provide theresults generated by knowledge engine 1130 to the module builder 1120.

Note that while shown as a single memory 1114 in FIG. 11 for simplicity,the various aspects stored in memory 1114 may be stored in differentphysical memories, but all accessible by CPU 1102 via internal dataconnections such as bus 1112. While not depicted, other aspects may beincluded in memory 1114.

Note that FIG. 11 is just one example of a processing system, and otherprocessing systems including fewer, additional, or alternative aspectsare possible consistent with this disclosure.

Example Computing Device

FIG. 12 depicts an example processing device 1200 that may perform themethods described herein. For example, the processing device 1200 can bea physical processing device such as a laptop, tablet, or smartphone,and is not limited to a single processing device that performs themethods herein.

Processing device 1200 includes a central processing unit (CPU) 1202connected to a data bus 1212. CPU 1202 is configured to processcomputer-executable instructions, e.g., stored in memory 1214, and tocause the processing device 1200 to perform methods described above. CPU1202 is included to be representative of a single CPU, multiple CPUs, asingle CPU having multiple processing cores, and other forms ofprocessing architecture capable of executing computer-executableinstructions.

Processing device 1200 further includes input/output (I/O) device(s)1208 and I/O device interfaces 1204, which allows processing device 1200to interface with input/output devices 1208, such as, for example, peninput, microphones, and other devices that allow for interaction withprocessing device 1200. Note that processing device 1200 may connectwith external I/O devices through physical and wireless connections(e.g., an external display device).

Processing device 1200 further includes a network interface 1206, whichprovides processing device 1200 with access to external network 1210 andthereby other processing devices.

Processing device 1200 further includes memory 1214, which in thisexample includes UI component 1216. UI component 1216 may display one ormore user interfaces each with one or more user interface views that mayreceive user input that may be used for defining a calculation graph,completeness graph, or client UI view for a module.

Note that while shown as a single memory 1214 in FIG. 12 for simplicity,the various aspects stored in memory 1214 may be stored in differentphysical memories, but all accessible by CPU 1202 via internal dataconnections such as bus 1212. While not depicted, other aspects may beincluded in memory 1214.

EXAMPLE CLAUSES

Implementation examples are described in the following numbered clauses:

Clause 1: A method for encoding rules defining a completeness of input,comprising: receiving a first input comprising one or more tuples,wherein a tuple of the one or more tuples comprises one or more fieldsassociated with an operation, one or more indicators, and one or moremodifiers; receiving a second input associated with the one or moretuples; providing, to a knowledge engine, the first input and the secondinput; receiving, from the knowledge engine, a result based on the firstinput and the second input; determining, based on the result, a firstsymbol associated with a first tuple of the one or more tuples; anddisplaying the first symbol, wherein the first symbol indicates whetherthe first tuple is complete.

Clause 2: The method of Clause 1, further comprising: determining that afirst list comprises a first subset of the one or more tuples;determining, based on the result, one or more aspects associated withthe first list; and displaying a second symbol indicating the one ormore aspects associated with the first list.

Clause 3: The method of any one of Clauses 1-2, wherein: a combinationof an indicator of the first tuple and a modifier of the first tupledefines a condition associated with the first tuple, and the methodfurther comprises: determining the second input does not satisfy thecondition associated with the first tuple; and receiving a third inputthat satisfies the condition associated with the first tuple.

Clause 4: The method of any one of Clauses 1-3, further comprisingdisplaying a first value of the second input associated with a field ofthe first tuple.

Clause 5: The method of Clause 2, wherein: determining, based on theresult, the one or more aspects associated with the first listcomprises: evaluating the first list based on the result; determiningthat a second list comprises a second subset of the one or more tuples;and determining to not evaluate the second list based on the evaluatingthe first list; and the method further comprises displaying a thirdsymbol indicating that the second list is not evaluated.

Clause 6: The method of Clause 5, wherein the third symbol furtherindicates whether the second list would be evaluated based on adifferent result.

Clause 7: The method of any one of Clauses 5-6, wherein: the first listcomprises the first tuple and a second tuple; determining, based on theresult, the one or more aspects associated with the first listcomprises: evaluating the first list based on the result by: determiningthat the second input contains required input to the first tuple; anddetermining that the second tuple does not require input based ondetermining that the second input contains the required input to thefirst tuple; and wherein the second symbol further indicates that thesecond tuple does not require input based on determining that the secondinput contains the required input to the first tuple.

Clause 8: The method of Clause 7, further comprising displaying a thirdsymbol indicating that the second input contains input to a field of thesecond tuple.

Clause 9: The method of any one of Clauses 2 and 7-8, further comprisingevaluating a second list comprising a second subset of the one or moretuples.

Clause 10: The method of any one of Clauses 7-9, further comprisingdetermining not to evaluate a second list based on the evaluating thefirst list.

Clause 11: The method of any one of Clauses 1-10, wherein the firstsymbol associated with the first tuple further indicates either that:the second input contains input to a field of the first tuple; or thesecond input does not contain input to the field of the first tuple.

Clause 12: The method of Clause 11, wherein the first symbol associatedwith the first tuple further indicates whether the field of the firsttuple requires input.

Clause 13: The method of any one of Clauses 1-12, wherein the firstsymbol associated with the first tuple further indicates that no tuplesof the one or more tuples are evaluated after determining, based on theresult, the first symbol associated with the first tuple.

Clause 14: The method of any one of Clauses 1-13, further comprising:determining, based on the result, that a second tuple did not need to beevaluated for the operation to be completed; and displaying a secondsymbol for a second tuple indicating that second tuple did not need tobe evaluated for the operation to be completed.

Clause 15: A processing system, comprising: a memory comprisingcomputer-executable instructions; one or more processors configured toexecute the computer-executable instructions and cause the processingsystem to perform a method in accordance with any one of Clauses 1-14.

Clause 16: A processing system, comprising means for performing a methodin accordance with any one of Clauses 1-14.

Clause 17: A non-transitory computer-readable medium comprisingcomputer-executable instructions that, when executed by one or moreprocessors of a processing system, cause the processing system toperform a method in accordance with any one of Clauses 1-14.

Clause 18: A computer program product embodied on a computer-readablestorage medium comprising code for performing a method in accordancewith any one of Clauses 1-14.

Additional Considerations

The preceding description is provided to enable any person skilled inthe art to practice the various embodiments described herein. Theexamples discussed herein are not limiting of the scope, applicability,or embodiments set forth in the claims. Various modifications to theseembodiments will be readily apparent to those skilled in the art, andthe generic principles defined herein may be applied to otherembodiments. For example, changes may be made in the function andarrangement of elements discussed without departing from the scope ofthe disclosure. Various examples may omit, substitute, or add variousprocedures or components as appropriate. For instance, the methodsdescribed may be performed in an order different from that described,and various steps may be added, omitted, or combined. Also, featuresdescribed with respect to some examples may be combined in some otherexamples. For example, an apparatus may be implemented or a method maybe practiced using any number of the aspects set forth herein. Inaddition, the scope of the disclosure is intended to cover such anapparatus or method that is practiced using other structure,functionality, or structure and functionality in addition to, or otherthan, the various aspects of the disclosure set forth herein. It shouldbe understood that any aspect of the disclosure disclosed herein may beembodied by one or more elements of a claim.

As used herein, the word “exemplary” means “serving as an example,instance, or illustration.” Any aspect described herein as “exemplary”is not necessarily to be construed as preferred or advantageous overother aspects.

As used herein, a phrase referring to “at least one of” a list of itemsrefers to any combination of those items, including single members. Asan example, “at least one of: a, b, or c” is intended to cover a, b, c,a-b, a-c, b-c, and a-b-c, as well as any combination with multiples ofthe same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b,b-b-b, b-b-c, c-c, and c-c-c or any other ordering of a, b, and c).

As used herein, the term “determining” encompasses a wide variety ofactions. For example, “determining” may include calculating, computing,processing, deriving, investigating, looking up (e.g., looking up in atable, a database or another data structure), ascertaining and the like.Also, “determining” may include receiving (e.g., receiving information),accessing (e.g., accessing data in a memory) and the like. Also,“determining” may include resolving, selecting, choosing, establishingand the like.

The methods disclosed herein comprise one or more steps or actions forachieving the methods. The method steps and/or actions may beinterchanged with one another without departing from the scope of theclaims. In other words, unless a specific order of steps or actions isspecified, the order and/or use of specific steps and/or actions may bemodified without departing from the scope of the claims. Further, thevarious operations of methods described above may be performed by anysuitable means capable of performing the corresponding functions. Themeans may include various hardware and/or software component(s) and/ormodule(s), including, but not limited to a circuit, an applicationspecific integrated circuit (ASIC), or processor. Generally, where thereare operations illustrated in figures, those operations may havecorresponding counterpart means-plus-function components with similarnumbering.

The following claims are not intended to be limited to the embodimentsshown herein, but are to be accorded the full scope consistent with thelanguage of the claims. Within a claim, reference to an element in thesingular is not intended to mean “one and only one” unless specificallyso stated, but rather “one or more.” Unless specifically statedotherwise, the term “some” refers to one or more. No claim element is tobe construed under the provisions of 35 U.S.C. § 112(f) unless theelement is expressly recited using the phrase “means for” or, in thecase of a method claim, the element is recited using the phrase “stepfor.” All structural and functional equivalents to the elements of thevarious aspects described throughout this disclosure that are known orlater come to be known to those of ordinary skill in the art areexpressly incorporated herein by reference and are intended to beencompassed by the claims. Moreover, nothing disclosed herein isintended to be dedicated to the public regardless of whether suchdisclosure is explicitly recited in the claims.

What is claimed is:
 1. A method for encoding rules defining acompleteness of input, comprising: receiving a first input comprisingone or more tuples, wherein a tuple of the one or more tuples comprisesone or more fields associated with an operation, one or more indicators,and one or more modifiers; receiving a second input associated with theone or more tuples; providing, to a knowledge engine, the first inputand the second input; receiving, from the knowledge engine, a resultbased on the first input and the second input; determining, based on theresult, a first symbol associated with a first tuple of the one or moretuples; and displaying the first symbol, wherein the first symbolindicates whether the first tuple is complete.
 2. The method of claim 1,further comprising: determining that a first list comprises a firstsubset of the one or more tuples; determining, based on the result, oneor more aspects associated with the first list; and displaying a secondsymbol indicating the one or more aspects associated with the firstlist.
 3. The method of claim 1, wherein: a combination of an indicatorof the first tuple and a modifier of the first tuple defines a conditionassociated with the first tuple, and the method further comprises:determining the second input does not satisfy the condition associatedwith the first tuple; and receiving a third input that satisfies thecondition associated with the first tuple.
 4. The method of claim 1,further comprising displaying a first value of the second inputassociated with a field of the first tuple.
 5. The method of claim 2,wherein: determining, based on the result, the one or more aspectsassociated with the first list comprises: evaluating the first listbased on the result; determining that a second list comprises a secondsubset of the one or more tuples; and determining to not evaluate thesecond list based on the evaluating the first list; and the methodfurther comprises displaying a third symbol indicating that the secondlist is not evaluated.
 6. The method of claim 5, wherein the thirdsymbol further indicates whether the second list would be evaluatedbased on a different result.
 7. The method of claim 2, wherein: thefirst list comprises the first tuple and a second tuple; determining,based on the result, the one or more aspects associated with the firstlist comprises: evaluating the first list based on the result by:determining that the second input contains required input to the firsttuple; and determining that the second tuple does not require inputbased on determining that the second input contains the required inputto the first tuple; and wherein the second symbol further indicates thatthe second tuple does not require input based on determining that thesecond input contains the required input to the first tuple.
 8. Themethod of claim 7, further comprising displaying a third symbolindicating that the second input contains input to a field of the secondtuple.
 9. The method of claim 7, further comprising evaluating a secondlist comprising a second subset of the one or more tuples.
 10. Themethod of claim 7, further comprising determining not to evaluate asecond list based on the evaluating the first list.
 11. The method ofclaim 1, wherein the first symbol associated with the first tuplefurther indicates either that: the second input contains input to afield of the first tuple; or the second input does not contain input tothe field of the first tuple.
 12. The method of claim 11, wherein thefirst symbol associated with the first tuple further indicates whetherthe field of the first tuple requires input.
 13. The method of claim 1,wherein the first symbol associated with the first tuple furtherindicates that no tuples of the one or more tuples are evaluated afterdetermining, based on the result, the first symbol associated with thefirst tuple.
 14. The method of claim 1, further comprising: determining,based on the result, that a second tuple did not need to be evaluatedfor the operation to be completed; and displaying a second symbol for asecond tuple indicating that second tuple did not need to be evaluatedfor the operation to be completed.
 15. A processing system, comprising:a memory storing executable instructions; and a processor configured toexecute the executable instructions and cause the processing system to:receive a first input comprising one or more tuples, wherein a tuple ofthe one or more tuples comprises one or more fields associated with anoperation, one or more indicators, and one or more modifiers; receive asecond input associated with the one or more tuples; provide, to aknowledge engine, the first input and the second input; receive, fromthe knowledge engine, a result based on the first input and the secondinput; determine, based on the result, a first symbol associated with afirst tuple of the one or more tuples; and display the first symbol,wherein the first symbol indicates whether the first tuple is complete.16. The processing system of claim 15, wherein the processor is furtherconfigured to cause the processing system to: determine that a firstlist comprises a first subset of the one or more tuples; determine,based on the result, one or more aspects associated with the first list;and display a second symbol indicating the one or more aspectsassociated with the first list.
 17. The processing system of claim 15,wherein: a combination of an indicator of the first tuple and a modifierof the first tuple defines a condition associated with the first tuple,and the processor is further configured to cause the processing systemto: determine the second input does not satisfy the condition associatedwith the first tuple; and receive a third input that satisfies thecondition associated with the first tuple.
 18. The processing system ofclaim 16, wherein: the first list comprises the first tuple and a secondtuple; wherein the processor being configured to cause the processingsystem to determine, based on the result, the one or more aspectsassociated with the first list comprises the processor being configuredto cause the processing system to: evaluate the first list based on theresult by: determining that the second input contains required input tothe first tuple; and determining that the second tuple does not requireinput based on determining that the second input contains the requiredinput to the first tuple; and wherein the second symbol furtherindicates that the second tuple does not require input based on thedetermining that the second input contains the required input to thefirst tuple.
 19. The processing system of claim 16, wherein: theprocessor being configured to cause the processing system to determine,based on the result, the one or more aspects associated with the firstlist comprises the processor being configured to cause the processingsystem to: evaluate the first list based on the result; determine that asecond list comprises a second subset of the one or more tuples; anddetermine to not evaluate the second list based on the evaluating thefirst list; and the processor is further configured to cause theprocessing system to display a third symbol indicating that the secondlist is not evaluated.
 20. A method for debugging a completeness of amodule, comprising: providing, to a user interface of a computingdevice, a first input comprising one or more tuples, wherein a tuple ofthe one or more tuples comprises one or more fields associated with anoperation of the module, one or more indicators, and one or moremodifiers; providing, to the user interface, a second input associatedwith the one or more tuples; receiving, at the user interface a firstsymbol associated with a first tuple of the one or more tuples, whereinthe first symbol indicates that the first tuple is not complete;providing, to the user interface, a third input after receiving thefirst symbol; and receiving, at the user interface, a second symbolassociated with the first tuple, wherein the second symbol indicatesthat the first tuple is complete.